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ABSTRACT 

A paper and pencil instrument was developed 
consisting of statements about teaching practices appropriate for 
students of ages 10 through 14- The statements are to be accepted or 
rejected by the respondent teacher* This paper reports the procedures 
and results of a third field test and the design of a fourth field 
test* The third field test instrument consisted of 244 items. It was 
administered to 320 teachers in eight schools. Two hundred of the 
items underwent statistical treatment: item analysis and factor 
analysis. The next step was to identify which of the 200 items 
correlated with data from classroom observations. Four observation 
instruments were used. The raw scores from the observations, combined 
into sets and converted into ratios were then correlated with the 
paper and pencil item responses of the 40 teachers. Five statistical 
criteria were computed by which to check the strength of each paper 
and oencil item. Through these criteria 97 items were eliminated and 
103 emerged. Of those that emerged: (1) Four- fifths of the items 

correlated both statistically and logically with one or more sets of 
observation data; (2) Forty-five of the items loaded in factors; (3) 
Twelve neither correlate with observation data nor load into factors 
but were retained because they discriminate well between high scorers 
and low scorers. When the instrument is fully developed it will be 
used as one part of a pre- training inventory procedure by which 
teacher trainees can obtain an estimate of their general competencies 
and a diagnosis as which to need development. Finally, systematic 
observation of classroom processes discriminates between teachers in 
many useful categories. (CK) 
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Paper and pencil instruments designed to measure beliefs or at- 
ti|u4«r are frequently used to aid in placing persons into jobs and 
are sometimes used for matching them with training situations. 

When such an instrument is used in that way v an assumption is made 
that the belief profile of the instrument is predictive of on-the- 
job behavior. The validity of that assumption has been tested in a 
variety of ways for different instruments; however, most such 
validity checks have been rather indirect. 

in the teaching profession, the Minnesota Teacher Attitude In- 
ventory is a case in point. During its development the MTA| was 
validated by the school administrator's judgment about the teacher, 
by pupil rating, and by rating of classroom observers according to 
some global categories of behavior. Systematic sampling of teacher 
behavior was not undertaken. 

Systematic classroom observation instrument developed in recent 
years make possible a more direct te^ln&of the linkage between 
teachers* belief statements and their orert teaching behavior. The 
research reported here is an effort to develop a paper and pencil 
instrument the responses to which are observable patterns of be- 
havior as recorded by systematic classroom observation instruments. 
We used four well-researched observation instruments and categories 
of our own. 
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The Instrument being developed . The paper and pencil instrument 
we have been developing consists of statements about teaching prac- 
tices appropriate for students of ages 10 through l4<. The statements 
are to be accepted or rejected by the respondent teacher. Below 
ar« two examples of items: 

a. Students seldom or never ask for clarif i cat ion of what is 
discut,3ed in ciasswork. 

b. When students analyze their own accomplishments the lear-* 
ning outcome is superior to that which is accomplished 
through teacher evaluation. 

The validation question concerning example A. above is whether the 
teacher who accepts that statement has a classroom in which fewer 
clarification questions are asked by students. And with example B. 
the validation question is whether the teacher who agrees that it is 
important for students to analyze their own learning outcomes does 
indeed have more student analyzing occurring in his classroom than does a 
teacher who rejects that item. 

* A presentation at AERA 1972 Annual Meeting, Session 11.3 
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The main task of our project is to produce paper and pencil 
items which meet several criteria, the chief one being corre- 
lation with systematic observation data. This paper reports the 
procedures and results of our third field test and the design of 
our fourth field test, now in progress. 

Procedures . The third field test instrument consisted of 244 
Items. It was administered to 320 teachers in 8 schools. Two 
hundred of the items underwent statistical treatment: item analysis 
and factor analysis. The factor analysis showed about half the items 
loaded into small factor sets, with 2 to 5 items per factor. Nine- 
teen factors were logically matched to the theoretical categories of 
itmes we started with. 

The next step was to identify which of the 200 items correlated 
with data from classroom observation. Four observation instruments 
were used: the Reciprocal Category System (Ober, 1968), the Florida 
Taxonomy of Cognitive Behavior (Brown, et al, 1967), the Teacher 
Practices Observation Record (Brown, 1968), and the Florida Climate 
and Control System (Soar, unpublished). Between them these systems 
categorize and tabulate some 300 different elements of teacher and 
student behavior — cognitive and affective, verbal and nonverbal, 
procedures and climate 

Two of the observation instruments are tallied “live, 11 and the 
other two (RCS and FTCB which register verbal behavior only) are 
tallied later from audio tapes made during the same observation se- 
quences . 

Twenty of the teachers whose responses agreed most with our 
scoring key and twenty teachers In ls^st agreement were identified to 
be observed. Neither the teams of trained observers nor the 
teachers themseives were aware of the basis of selection of teachers. 
The teachers were informed that we were observing to see whether 
teaching styles related in some ways to the paper and pencil instru- 
ment. Each teacher was observed 12 times over a period of several 
days. Each observation had a duration of 5 minutes. The observations 
were made in sets of three, on four different occasions. Previous 
research using these observation instruments, mainly the research of 
the'authors of the Instruments, supports the observation schedule we 
used as fully adequate to obtain a representative sample of the 
behavior patterns of those classrooms. 

The raw scores from the observations, combined into sets and 
converted into ratios as shown in Appendix A, were then correlated 
with the paper and pencil Item responses of those 40 teachers. 

Results of the statistical treatment . We had computed five 
statistical criteria by which to check the strength of each paper 
and pencil item: biserial r and discrimination index to check 
item correlation with total score on the instrument; difficulty 
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index; loading in a factor; and. .correlation with observation data. 

Through these criteria, 97 Items were eliminated. Below is a 
summary profile of the 103 items which emerged: 

(a) Four-fifths of the items correlated both statistically and 
logically with one or more sets of observation data. Cor- 
relations range from .30 to .60. In effect, the attitude, 
disposition or mind-set represented in the response to each 
Item has a counterpart in an observable behavior pattern in the 
classroom. Our procedure for selecting teachers to be 
observed appears to be well justified. The twenty teachers 
whose responses on the paper and pencil instrument most 
agreed with our scoring key also ranked as high scorers on 

the observation data; likewise the 20 who disagreed most had 
low scores on the observation keys, with a significant 
difference between the mean observation scores of the two 
groups (.001). 

(b) Forty-five of the 103 items loaded In factors. A total 

of 19 factors emerged which logically match the theoretical 
categories of items. 

(c) Twelve of the 103 items neither correlate with observation 

data nor load into factors but were retained because they / 

discriminate well between high scorers and low scorers. 

(d) A reliability estimate of .80 (Spearman-Brown Prophecy 
Formula) for the total 103 items was obtained by creating 
equivalent halves. That correlation is acceptable for this 

type of instrument. ^ 

-'a 

Design of current research on the paper and pencil i nstrument * 

When the instrument is fully developed it will be used as one part of 
a pre-training inventory procedure by which teacher trainees can obtain 
an estimate of their general competencies and a diagnosis as to 
which competencies need development. The instrunent now being 
field tested contains items which relate to 22 theoretical competency 
sets. When the task is finished, we expect to have the means to 
prepare for each respondent to the instrument a profile giving a 
score on each of the 22 competency sets. Now being tested are 480 
items, with a population of 950 teachers. If half of those items 
survive the statistical treatment, we will have enough for the com- 
pleted instrument. Procedures similar to those described above 
are being used, except this time the total score on the instrument 
will not be used; rather, scores of each of the 22 competency sets 
will be used. 

We are also in the process of writing some new observation 
categories. The four observation instruments used in the third 
field test need to be supplemented by additional observation cate- 
gories if we are to obtain adequate coverage for all 22 comptency 
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areas. It Is not especially difficult to generate new observation 
categories. A category is regarded as reliable if inter-observer 
agreement of .85 or better is consistently obtained. 

A final about the value of systematic observation of classroom 
processes: our experience shows that systematic observation dis- 
criminates between teachers in many useful categories. Until 
such time as research shows important correlations between teacher 
behavior and long term pupil growth, systematic observation of teacher 
influence on pupil classroom behavior is an especially valuable 
tool for teacher educators. 
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APPENDIX A 



Observation Elements 



The Reciprocal Category System includes Elements I through VI 

Element Is deals with the warm-cool subscale of the RCS and 

expresses the ratio of warm teacher behavior, 

cool 

■n-, — TT. e vtf+.b the accent-correct subscale of the RCS . 

Element II. the ratio of accent teacher behavior. 

. ■ ^ correct 

ttt • th the indirect-direct subscale of the RCS 

Element m. a . the ratio of amplifies teacher behavior. 



directs 



Element IV: expresses the warm, ratio of student behavior. 

cool 

Element V: expresses the accent ratio of student behavior. 

correct , 

Element VI: exnresses the indirect ratio of student behavior. 

direct 

The Florida Taxonomy of Cognitive Behavior includes Elements VII 
and VIII 

Element VII: is the ratio of teacher- behavior in category 1 oi 

the taxonomy (knowledge) over teacner behavior m 
categories 2* 3, 4, 5, 6 and 7* 

Teacher behavior _l_i_ 

Teacher behavior 

Element VIII: * s the ratio of student behavior in category 1 
Element vm. -s ^ne QVer student be havior in categories 

2,3, 4,5, 6 and ?. 

Student Behavior 1 
Student Behavior 4&52s62e7 



The TPOR (Teacher Practice Observation Record) includes Elemen 
XI through XV. 
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Element XI: 



refers to the nature of the situation (A) in .the 
ITCH, and expresses the ratio of experimental 
(2— 4-6-o-10)~ to non-experimental (1-3-5-7-9) 
teacher behavior. 



even T, 



total 
total odd !• 



behaviors 

behaviors 



Element XII: refers to, the nature of_the Hg&gg Jf) . Jev^og^ert 

- - - ' • se of subject matter CD) -- 



of ideas (C) and the 



of experimental 



the TECH. It expresses the ratio 
to non— experimental teacher behaviors in these 
categories 



total even T. behaviors, 
total odd T. behaviors 

Element XIII: refers to the evaluation (E) category in the TPOH 

and expresses the ratio of experimental teacher 

behavior 



total even T. behavio rs 
total odd T. behaviors 

Element XIV: refers to *the differentiation (F) category^of the 

TPOH, and expresses the ratio of experimental to 
non- experimental teacher behavior . 

total even T. behaviors 
total odd T. behaviors 

Element XV: jefers to the motivation-control (G) category of 

the TPOH, and expresses the ratio 

total even T. behaviors 
total oda T. behaviors 



The Florida Climate and Control System includes Elements XVI through 



expresses the ratio of nunil limite d choice in the 
FLACCS pupil no choice 

refers to teacher positive verbal control in tne^ 
FLACCS and expresses the amount of teacher positive 
verbal control 

actual T. positive control verb al behaviors, 
total number of observation cells in this group 



aa. 

Element XVI: 
Element XVII: 
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Element XVIIL : 
Element XIX: 

Element XX: 

The following 
defined by the 

Element XXI: 
Element XXII: 
Element XXIII; 

Element XXIV: 



refers to teacher negative verbal control behaviors 
in the FLACC3 

actual T, negative control verbal behaviors 
total number "of 1 observation cells in this group 

deals with teacher non-verbal control behaviors 
in the FLACCS and expresses the ratio of positive 
to negative teacher non-verbal control behaviors 



total positive te acher non-verbal control behavior; 
total negative teacher non-verbal control behav: or; 



expresses the ratio: of positive teacher affective 
--(climate) behavior over negative teacher affect in 
the FLACCS. 

total positive verbal and non-verbal teacher behaviors 
total negative verbal and non-verbal teacher behaviors 



six elements (XXI to XXVI) deal with specific behaviors 
researchers. ’ 



expresses the amount of teacher contact with other 
teachers or resource persons 

actual observed contacts 
total number of contact cells 

deals with the amount of student interest when 
working in small groups 

actual observed behavior 
total number of behavior cells 

deals with the amount of student distraction when 
working in small groups 

actual observed behaviors 
total number of behavior cells 

deals with the amount of student disruption v/hen 
working in small groups 

actual observed behaviors 
total number of behavior cells 



Element XXV : 



deals with the amount of teacher to student 
socialization 

actual observed behaviors 
total number of behavior cells 

Element XXVI: deals with the types of instructional materials 

used in the classroom 

total materials used 

total number of materials cells 



For Elements VII, VIII, XVIII, XXIII and XXIV a favorable score 
is a negative one. That is, negative scores are expected to 
correlate with the paper and pencil instrument. 
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